Natural Language Processing Research Research Articles

Neural Methods for Data-to-text Generation

The neural boom that has sparked natural language processing (NLP) research throughout the last decade has similarly led to significant innovations in data-to-text generation (D2T). This survey offers a consolidated view into the neural D2T paradigm with a structured examination of the approaches, benchmark datasets, and evaluation protocols. This survey draws boundaries separating D2T from the rest of the natural language generation (NLG) landscape, encompassing an up-to-date synthesis of the literature, and highlighting the stages of technological adoption from within and outside the greater NLG umbrella. With this holistic view, we highlight promising avenues for D2T research that not only focus on the design of linguistically capable systems but also systems that exhibit fairness and accountability.

Multi-Label Text Classification model integrating label attention and historical attention

Multi-Label Text Classification (MLTC) is one of the most import research of natural language processing. Although Deep Learning (DL) models have been widely applied to MLTC, there still exist some drawbacks. First, traditional DL models utilize all the words in the document to construct the embedding vector, while there are many words that affect the classification results. Next, the labels in MLTC have specific semantics, while traditional DL models ignore fine-grained matching signals between words and labels. Then, traditional DL models have difficulty in handling the data imbalance issue in MLTC datasets. In addition, during the training process, small errors in a certain epoch may be amplified with the increase of the number of iterations, resulting in classification errors. To address the above problems, a MLTC model integrating Label Attention and Historical Attention (i.e. LAHA) is proposed. First, a word filter is set up to select important words based on the cosine similarity between words and labels. Next, Document Self Attention (DSA) and Label Attention (LA) are obtained, and DSA-attended LA co-attention (LA-co) and LA-attended DSA co-attention (DAS-co) networks are constructed. Then, the fine-grained matching signals between words and labels are integrated through the adaptive plus of LA-co and DAS-co. At last, Historical Attention is integrated to LAHA, which not only avoids mis-classification caused by minor errors of a certain epoch, but also reduces overfitting to high-frequency labels. Multiple comparative experiments on four benchmark datasets demonstrate that LAHA outperforms the state-of-the-art baseline models and can effectively solve the data imbalance issue in MLTC datasets. Our code is available at https://github.com/sgysgywaityou/LAHA.

BERT-based language model for accurate drug adverse event extraction from social media: implementation, evaluation, and contributions to pharmacovigilance practices.

Social media platforms serve as a valuable resource for users to share health-related information, aiding in the monitoring of adverse events linked to medications and treatments in drug safety surveillance. However, extracting drug-related adverse events accurately and efficiently from social media poses challenges in both natural language processing research and the pharmacovigilance domain. Recognizing the lack of detailed implementation and evaluation of Bidirectional Encoder Representations from Transformers (BERT)-based models for drug adverse event extraction on social media, we developed a BERT-based language model tailored to identifying drug adverse events in this context. Our model utilized publicly available labeled adverse event data from the ADE-Corpus-V2. Constructing the BERT-based model involved optimizing key hyperparameters, such as the number of training epochs, batch size, and learning rate. Through ten hold-out evaluations on ADE-Corpus-V2 data and external social media datasets, our model consistently demonstrated high accuracy in drug adverse event detection. The hold-out evaluations resulted in average F1 scores of 0.8575, 0.9049, and 0.9813 for detecting words of adverse events, words in adverse events, and words not in adverse events, respectively. External validation using human-labeled adverse event tweets data from SMM4H further substantiated the effectiveness of our model, yielding F1 scores 0.8127, 0.8068, and 0.9790 for detecting words of adverse events, words in adverse events, and words not in adverse events, respectively. This study not only showcases the effectiveness of BERT-based language models in accurately identifying drug-related adverse events in the dynamic landscape of social media data, but also addresses the need for the implementation of a comprehensive study design and evaluation. By doing so, we contribute to the advancement of pharmacovigilance practices and methodologies in the context of emerging information sources like social media.

An Empirical Evaluation of Prompting Strategies for Large Language Models in Zero-Shot Clinical Natural Language Processing: Algorithm Development and Validation Study.

Large language models (LLMs) have shown remarkable capabilities in natural language processing (NLP), especially in domains where labeled data are scarce or expensive, such as the clinical domain. However, to unlock the clinical knowledge hidden in these LLMs, we need to design effective prompts that can guide them to perform specific clinical NLP tasks without any task-specific training data. This is known as in-context learning, which is an art and science that requires understanding the strengths and weaknesses of different LLMs and prompt engineering approaches. The objective of this study is to assess the effectiveness of various prompt engineering techniques, including 2 newly introduced types-heuristic and ensemble prompts, for zero-shot and few-shot clinical information extraction using pretrained language models. This comprehensive experimental study evaluated different prompt types (simple prefix, simple cloze, chain of thought, anticipatory, heuristic, and ensemble) across 5 clinical NLP tasks: clinical sense disambiguation, biomedical evidence extraction, coreference resolution, medication status extraction, and medication attribute extraction. The performance of these prompts was assessed using 3 state-of-the-art language models: GPT-3.5 (OpenAI), Gemini (Google), and LLaMA-2 (Meta). The study contrasted zero-shot with few-shot prompting and explored the effectiveness of ensemble approaches. The study revealed that task-specific prompt tailoring is vital for the high performance of LLMs for zero-shot clinical NLP. In clinical sense disambiguation, GPT-3.5 achieved an accuracy of 0.96 with heuristic prompts and 0.94 in biomedical evidence extraction. Heuristic prompts, alongside chain of thought prompts, were highly effective across tasks. Few-shot prompting improved performance in complex scenarios, and ensemble approaches capitalized on multiple prompt strengths. GPT-3.5 consistently outperformed Gemini and LLaMA-2 across tasks and prompt types. This study provides a rigorous evaluation of prompt engineering methodologies and introduces innovative techniques for clinical information extraction, demonstrating the potential of in-context learning in the clinical domain. These findings offer clear guidelines for future prompt-based clinical NLP research, facilitating engagement by non-NLP experts in clinical NLP advancements. To the best of our knowledge, this is one of the first works on the empirical evaluation of different prompt engineering approaches for clinical NLP in this era of generative artificial intelligence, and we hope that it will inspire and inform future research in this area.

Enabling Natural Language Processing and AI Research in Low-Resource Languages: Development and Description of an Assamese UPoS Tagged Dataset

This paper describes in detail the Universal Parts of Speech (UPoS) tagged dataset for the Assamese language. PoS tagged dataset in a language is crucial for experimenting and creating resources for various Natural Language Processing (NLP) and AI research. With the growing usage of Universal Dependency standards, tagged dataset with Universal PoS tags are becoming very much essential for contemporary experiments in the NLP community. NLP research in Assamese, and Indo-Aryan language, is relatively new, and the language is considered a Low Resource language. The dataset of UPoS tagged Assamese text is created with an aim of contributing towards enriching this low resource language for NLP and AI tasks. The size of the dataset is 283506 tokens of Assamese vocabulary, against total 20280 sentences, tagged with 17 standard UPoS tags of core lexical categories. The raw data are taken from an open-source corpus originally tagged with BIS tagset. The original size of 453457 tokens against 29504 sentences, after subjected to data filtering, was reduced to this clean resource of 283506 tokens. Lexical categories mapping is done with linguistic expertise, from BIS to UPoS tagsets. Mapped pattern was used for a first-level conversion of BIS tags to UPoS tags. Linguistic validation is also performed with linguistic experts and inter annotator agreement/disagreements were recorded. Second level validation resulted in deciding on the agreement, producing the final version of the dataset. This Assamese UPoS tagged dataset is the first of its kind with UPoS annotations and shall serve a wider Assamese NLP research community for model training using Machine Learning/Deep Learning Techniques.

A Survey on Automatic Generation of Figurative Language: From Rule-based Systems to Large Language Models

Figurative language generation (FLG) is the task of reformulating a given text to include a desired figure of speech, such as a hyperbole, a simile, and several others, while still being faithful to the original context. This is a fundamental, yet challenging task in Natural Language Processing (NLP), which has recently received increased attention due to the promising performance brought by pre-trained language models. Our survey provides a systematic overview of the development of FLG, mostly in English, starting with the description of some common figures of speech, their corresponding generation tasks and datasets. We then focus on various modelling approaches and assessment strategies, leading us to discussing some challenges in this field, and suggesting some potential directions for future research. To the best of our knowledge, this is the first survey that summarizes the progress of FLG including the most recent development in NLP. We also organize corresponding resources, e.g., paper lists and datasets, and make them accessible in an open repository. We hope this survey can help researchers in NLP and related fields to easily track the academic frontier, providing them with a landscape and a roadmap of this area.

Sentiment analysis of COP9-related tweets: a comparative study of pre-trained models and traditional techniques.

Sentiment analysis has become a crucial area of research in natural language processing in recent years. The study aims to compare the performance of various sentiment analysis techniques, including lexicon-based, machine learning, Bi-LSTM, BERT, and GPT-3 approaches, using two commonly used datasets, IMDB reviews and Sentiment140. The objective is to identify the best-performing technique for an exemplar dataset, tweets associated with the WHO Framework Convention on Tobacco Control Ninth Conference of the Parties in 2021 (COP9). A two-stage evaluation was conducted. In the first stage, various techniques were compared on standard sentiment analysis datasets using standard evaluation metrics such as accuracy, F1-score, and precision. In the second stage, the best-performing techniques from the first stage were applied to partially annotated COP9 conference-related tweets. In the first stage, BERT achieved the highest F1-scores (0.9380 for IMDB and 0.8114 for Sentiment 140), followed by GPT-3 (0.9119 and 0.7913) and Bi-LSTM (0.8971 and 0.7778). In the second stage, GPT-3 performed the best for sentiment analysis on partially annotated COP9 conference-related tweets, with an F1-score of 0.8812. The study demonstrates the effectiveness of pre-trained models like BERT and GPT-3 for sentiment analysis tasks, outperforming traditional techniques on standard datasets. Moreover, the better performance of GPT-3 on the partially annotated COP9 tweets highlights its ability to generalize well to domain-specific data with limited annotations. This provides researchers and practitioners with a viable option of using pre-trained models for sentiment analysis in scenarios with limited or no annotated data across different domains.

Enhancing Natural Language Processing (NLP) through VIKOR Method: A Comprehensive Approach for Improved Computational Linguistics

Natural Language Processing (NLP) is a revolutionary discipline that resides at the crossroads of computer science and linguistics, with its primary emphasis on the interaction between computers and human language. With roots in artificial intelligence, NLP seeks to equip machines with the ability to comprehend, interpret, and respond to natural language, enabling more intuitive and meaningful communication between humans and computers. This multidisciplinary domain leverages advanced algorithms and models to tackle a range of linguistic challenges, from language translation and sentiment analysis to speech recognition. Recent breakthroughs, particularly in deep learning and neural networks, have propelled NLP to new heights, with applications spanning diverse sectors such as healthcare, finance, and education. As NLP continues to advance, its potential impact on enhancing human-machine interaction and information processing is increasingly evident, promising a future where technology seamlessly integrates with our natural modes of communication. The significance of Natural Language Processing (NLP) in research lies in its capacity to revolutionize human-computer interaction. NLP empowers machines to understand and generate human language, enabling advancements in sentiment analysis, language translation, and conversational AI. This transformative capability has profound implications across various industries, including healthcare, finance, and education. NLP research not only enhances the efficiency of information retrieval and processing but also holds the key to developing more intuitive and user-friendly technologies. As the frontier of NLP research expands, its potential to bridge the communication gap between humans and machines continues to shape the future of technology and information access. VIKOR functions as a multi-criteria decision-making technique that determines the best option by comparing it to the ideal option. The ranking procedure comprises determining distances concerning the optimal solution. VIKOR uses linear normalization to get the best possible outcomes. This technique was first presented by Opricovic in 1998 and was intended to optimize multi-attribute complex systems. Its main focus is on ranking lists that allow for flexibility in the strategy weight interval and incorporate compromise alternatives to obtain desired results. Alternative taken as Open AI GPT-4, Google BERT, Microsoft Azure Text Analytics, IBM Watson Natural Language Understanding, spaCy, NLTK (Natural Language Toolkit), Amazon Comprehend. Evaluation Parameter taken as Sentiment Analysis (F1 Score), NER Performance, Language Support, Processing Speed (Response Time) (ms), Translation Accuracy (BLEU Score), Data Privacy. The result it is seen that Amazon Comprehend is got the first rank where as is the IBM Watson Natural Language Understanding is having the lowest rank.

Exploring the potential of ChatGPT in medical dialogue summarization: a study on consistency with human preferences

BackgroundTelemedicine has experienced rapid growth in recent years, aiming to enhance medical efficiency and reduce the workload of healthcare professionals. During the COVID-19 pandemic in 2019, it became especially crucial, enabling remote screenings and access to healthcare services while maintaining social distancing. Online consultation platforms have emerged, but the demand has strained the availability of medical professionals, directly leading to research and development in automated medical consultation. Specifically, there is a need for efficient and accurate medical dialogue summarization algorithms to condense lengthy conversations into shorter versions focused on relevant medical facts. The success of large language models like generative pre-trained transformer (GPT)-3 has recently prompted a paradigm shift in natural language processing (NLP) research. In this paper, we will explore its impact on medical dialogue summarization.MethodsWe present the performance and evaluation results of two approaches on a medical dialogue dataset. The first approach is based on fine-tuned pre-trained language models, such as bert-based summarization (BERTSUM) and bidirectional auto-regressive Transformers (BART). The second approach utilizes a large language models (LLMs) GPT-3.5 with inter-context learning (ICL). Evaluation is conducted using automated metrics such as ROUGE and BERTScore.ResultsIn comparison to the BART and ChatGPT models, the summaries generated by the BERTSUM model not only exhibit significantly lower ROUGE and BERTScore values but also fail to pass the testing for any of the metrics in manual evaluation. On the other hand, the BART model achieved the highest ROUGE and BERTScore values among all evaluated models, surpassing ChatGPT. Its ROUGE-1, ROUGE-2, ROUGE-L, and BERTScore values were 14.94%, 53.48%, 32.84%, and 6.73% higher respectively than ChatGPT’s best results. However, in the manual evaluation by medical experts, the summaries generated by the BART model exhibit satisfactory performance only in the “Readability” metric, with less than 30% passing the manual evaluation in other metrics. When compared to the BERTSUM and BART models, the ChatGPT model was evidently more favored by human medical experts.ConclusionOn one hand, the GPT-3.5 model can manipulate the style and outcomes of medical dialogue summaries through various prompts. The generated content is not only better received than results from certain human experts but also more comprehensible, making it a promising avenue for automated medical dialogue summarization. On the other hand, automated evaluation mechanisms like ROUGE and BERTScore fall short in fully assessing the outputs of large language models like GPT-3.5. Therefore, it is necessary to research more appropriate evaluation criteria.

Sentiment analysis model for cryptocurrency tweets using different deep learning techniques

Abstract Bitcoin (BTC) is one of the most important cryptocurrencies widely used in various financial and commercial transactions due to the fluctuations in the price of this currency. Recent research in large data analytics and natural language processing has resulted in the development of automated techniques for assessing the sentiment in online communities, which has emerged as a crucial platform for users to express their thoughts and comments. Twitter, one of the most well-known social media platforms, provides many tweets about the BTC cryptocurrency. With this knowledge, we can apply deep learning (DL) to use these data to predict BTC price variations. The researchers are interested in studying and analyzing the reasons contributing to the BTC price’s erratic movement by analyzing Twitter sentiment. The main problem in this article is that no standard model with high accuracy can be relied upon in analyzing textual emotions, as it represents one of the factors affecting the rise and fall in the price of cryptocurrencies. This article aims to classify the sentiments of an expression into positive, negative, or neutral emotions. The methods that have been used are word embedding FastText model in addition to different DL methods that deal with time series, one-dimensional convolutional neural networks (CONV1D), long-short-term memory networks (LSTMs), recurrent neural networks, gated recurrent units, and a Bi-LSTM + CONV1D The main results revealed that the LSTM method, based on the DL technique, achieved the best results. The performance accuracy of the methods was 95.01, 95.95, 80.59, 95.82, and 95.67%, respectively. Thus, we conclude that the LSTM method achieved better results than other methods in analyzing the textual sentiment of BTC.

Towards Mental Health Analysis in Social Media for Low-resourced Languages

The surge in internet use for expression of personal thoughts and beliefs has made it increasingly feasible for the social Natural Language Processing (NLP) research community to find and validate associations between social media posts and mental health status . Cross-sectional and longitudinal studies of low-resourced social media data bring to fore the importance of real-time responsible Artificial Intelligence (AI) models for mental health analysis in native languages. Aiming at classifying research for social computing and tracking advances in the development of learning-based models, we propose a comprehensive survey on mental health analysis for social media and posit the need of analyzing low-resourced social media data for mental health . We first classify three components for computing on social media as: SM - data mining/natural language processing on social media , IA - integrated applications with social media data and user-network modeling, and NM - user and network modeling on social networks. To this end, we posit the need of mental health analysis in different languages of East Asia (e.g., Chinese, Japanese, Korean), South Asia (Hindi, Bengali, Tamil), Southeast Asia (Malay, Thai, Vietnamese), European languages (Spanish, French) and the Middle East (Arabic). Our comprehensive study examines available resources and recent advances in low-resourced languages for different aspects of SM, IA, and NM to discover new frontiers as potential field of research.

Text‐based sentiment analysis in finance: Synthesising the existing literature and exploring future directions

SummaryAdvances in Deep Learning have drastically improved the abilities of Natural Language Processing (NLP) research, creating new state‐of‐the‐art benchmarks. Two research streams at the forefront of NLP analysis are transformer architecture and multimodal analysis. This paper critically evaluates the extant literature applying sentiment analysis techniques to the financial domain. We classify the financial sentiment analysis literature according to the most used techniques in the area, with a focus on methods used to detect sentiment within corporate earnings conference calls, because of their dual modality (text‐audio) nature. We find that the financial literature follows a similar path to NLP sentiment literature, in that more advanced techniques to define sentiment are being used as the field progresses. However, techniques used to determine financial sentiment currently fall behind state‐of‐the‐art techniques used within NLP. Two future directions stem from this paper. Firstly, we propose that the adoption of transformer architecture to create robust representations of textual data could enhance sentiment analysis in academic finance. Secondly, the adoption of multimodal classifiers in finance represents a new, currently underexplored area of study that offers opportunities for finance research.

TCGA-Reports: A machine-readable pathology report resource for benchmarking text-based AI models

In cancer research, pathology report text is a largely untapped data source. Pathology reports are routinely generated, more nuanced than structured data, and contain added insight from pathologists. However, there are no publicly available datasets for benchmarking report-based models. Two recent advances suggest the urgent need for a benchmark dataset. First, improved optical character recognition (OCR) techniques will make it possible to access older pathology reports in an automated way, increasing the data available for analysis. Second, recent improvements in natural language processing (NLP) techniques using artificial intelligence (AI) allow more accurate prediction of clinical targets from text. We apply state-of-the-art OCR and customized post-processing to report PDFs from The Cancer Genome Atlas, generating a machine-readable corpus of 9,523 reports. Finally, we perform a proof-of-principle cancer-type classification across 32 tissues, achieving 0.992 average AU-ROC. This dataset will be useful to researchers across specialties, including research clinicians, clinical trial investigators, and clinical NLP researchers.

A Survey on Challenges and Advances in Natural Language Processing with a Focus on Legal Informatics and Low-Resource Languages

The field of Natural Language Processing (NLP) has experienced significant growth in recent years, largely due to advancements in Deep Learning technology and especially Large Language Models. These improvements have allowed for the development of new models and architectures that have been successfully applied in various real-world applications. Despite this progress, the field of Legal Informatics has been slow to adopt these techniques. In this study, we conducted an extensive literature review of NLP research focused on legislative documents. We present the current state-of-the-art NLP tasks related to Law Consolidation, highlighting the challenges that arise in low-resource languages. Our goal is to outline the difficulties faced by this field and the methods that have been developed to overcome them. Finally, we provide examples of NLP implementations in the legal domain and discuss potential future directions.

Multi-aspect multilingual and cross-lingual parliamentary speech analysis

Parliamentary and legislative debate transcripts provide an informative insight into elected politicians’ opinions, positions, and policy preferences. They are interesting for political and social sciences as well as linguistics and natural language processing (NLP) research. While exiting research studied individual parliaments, we apply advanced NLP methods to a joint and comparative analysis of six national parliaments (Bulgarian, Czech, French, Slovene, Spanish, and United Kingdom) between 2017 and 2020. We analyze emotions and sentiment in the transcripts from the ParlaMint dataset collection, and assess if the age, gender, and political orientation of speakers can be detected from their speeches. The results show some commonalities and many surprising differences among the analyzed countries.

A hybrid combination of CNN Attention with optimized random forest with grey wolf optimizer to discriminate between Arabic hateful, abusive tweets

Arabic hateful speech recognition has long been a major area of focus in Natural Language Processing (NLP) research. In light of recent advancements in transformer models and deep learning, researchers are now turning to transfer learning techniques based on existing models such as BERT for Arabic hateful speech recognition. To detect Arabic hateful contexts, using advanced machine learning algorithms and NLP techniques is essential. These techniques can help to detect different forms of hateful contexts in Arabic by analyzing the text for lexical, semantic, and syntactic features. In this research, we proposed a new hybrid approach that combines deep and machine learning models to detect hateful and abusive content in Arabic. The proposed model consists of a combination of convolutional neural networks and attention layers that are trained to differentiate between normal, abusive, and hateful contexts in Arabic. In the first step, we used a pre-trained model to extract features from the hateful Arabic context. After that, we used an optimized random forest combined with particle swarm optimization and grey wolf optimizer to classify the extracted features. Finally, we evaluated the performance of the model to detect hateful Arabic contexts. To evaluate the proposed method we used 5846 and 6023 tweets with 3 categories of hateful, abusive, and normal Arabic contexts. The experimental result indicates 97.16% accuracy, 97.15% F1-score, 97.17% precision, and 97.13% sensitivity using CNN Attention + optimized random forest by the grey wolf optimizer on 5846 tweets. 97.83% accuracy, 97.83% F1-score, 97.84% precision, and 97.83% sensitivity have been reported CNN Attention + optimized random forest by the grey wolf optimizer on 6023 tweets.

The Role of Typological Feature Prediction in NLP and Linguistics

Abstract Computational typology has gained traction in the field of Natural Language Processing (NLP) in recent years, as evidenced by the increasing number of papers on the topic and the establishment of a Special Interest Group on the topic (SIGTYP), including the organization of successful workshops and shared tasks. A considerable amount of work in this sub-field is concerned with prediction of typological features, for example, for databases such as the World Atlas of Language Structures (WALS) or Grambank. Prediction is argued to be useful either because (1) it allows for obtaining feature values for relatively undocumented languages, alleviating the sparseness in WALS, in turn argued to be useful for both NLP and linguistics; and (2) it allows us to probe models to see whether or not these typological features are encapsulated in, for example, language representations. In this article, we present a critical stance concerning prediction of typological features, investigating to what extent this line of research is aligned with purported needs—both from the perspective of NLP practitioners, and perhaps more importantly, from the perspective of linguists specialized in typology and language documentation. We provide evidence that this line of research in its current state suffers from a lack of interdisciplinary alignment. Based on an extensive survey of the linguistic typology community, we present concrete recommendations for future research in order to improve this alignment between linguists and NLP researchers, beyond the scope of typological feature prediction.

MSCAN: multi-scale self- and cross-attention network for RNA methylation site prediction

BackgroundEpi-transcriptome regulation through post-transcriptional RNA modifications is essential for all RNA types. Precise recognition of RNA modifications is critical for understanding their functions and regulatory mechanisms. However, wet experimental methods are often costly and time-consuming, limiting their wide range of applications. Therefore, recent research has focused on developing computational methods, particularly deep learning (DL). Bidirectional long short-term memory (BiLSTM), convolutional neural network (CNN), and the transformer have demonstrated achievements in modification site prediction. However, BiLSTM cannot achieve parallel computation, leading to a long training time, CNN cannot learn the dependencies of the long distance of the sequence, and the Transformer lacks information interaction with sequences at different scales. This insight underscores the necessity for continued research and development in natural language processing (NLP) and DL to devise an enhanced prediction framework that can effectively address the challenges presented.ResultsThis study presents a multi-scale self- and cross-attention network (MSCAN) to identify the RNA methylation site using an NLP and DL way. Experiment results on twelve RNA modification sites (m6A, m1A, m5C, m5U, m6Am, m7G, Ψ, I, Am, Cm, Gm, and Um) reveal that the area under the receiver operating characteristic of MSCAN obtains respectively 98.34%, 85.41%, 97.29%, 96.74%, 99.04%, 79.94%, 76.22%, 65.69%, 92.92%, 92.03%, 95.77%, 89.66%, which is better than the state-of-the-art prediction model. This indicates that the model has strong generalization capabilities. Furthermore, MSCAN reveals a strong association among different types of RNA modifications from an experimental perspective. A user-friendly web server for predicting twelve widely occurring human RNA modification sites (m6A, m1A, m5C, m5U, m6Am, m7G, Ψ, I, Am, Cm, Gm, and Um) is available at http://47.242.23.141/MSCAN/index.php.ConclusionsA predictor framework has been developed through binary classification to predict RNA methylation sites.

Few-shot learning for automated content analysis: Efficient coding of arguments and claims in the debate on arms deliveries to Ukraine

Pre-trained language models (PLM) based on transformer neural networks developed in the field of natural language processing (NLP) offer great opportunities to improve automatic content analysis in communication science, especially for the coding of complex semantic categories in large datasets via supervised machine learning. However, three characteristics so far impeded the widespread adoption of the methods in the applying disciplines: the dominance of English language models in NLP research, the necessary computing resources, and the effort required to produce training data to fine-tune PLMs. In this study, we address these challenges by using a multilingual transformer model in combination with the adapter extension to transformers, and few-shot learning methods. We test our approach on a realistic use case from communication science to automatically detect claims and arguments together with their stance in the German news debate on arms deliveries to Ukraine. In three experiments, we evaluate (1) data preprocessing strategies and model variants for this task, (2) the performance of different few-shot learning methods, and (3) how well the best setup performs on varying training set sizes in terms of validity, reliability, replicability and reproducibility of the results. We find that our proposed combination of transformer adapters with pattern exploiting training provides a parameter-efficient and easily shareable alternative to fully fine-tuning PLMs. It performs on par in terms of validity, while overall, provides better properties for application in communication studies. The results also show that pre-fine-tuning for a task on a near-domain dataset leads to substantial improvement, in particular in the few-shot setting. Further, the results indicate that it is useful to bias the dataset away from the viewpoints of specific prominent individuals.

Deep Aspect-Sentinet: Aspect Based Emotional Sentiment Analysis Using Hybrid Attention Deep Learning Assisted BILSTM

Data mining and natural language processing researchers have been working on sentiment analysis for the past decade. Using deep neural networks (DNNs) for sentiment analysis has recently shown promising results. A technique of studying people’s attitudes through emotional sentiment analysis of data generated from various sources such as Twitter, social media reviews, etc. and classifying emotions based on the given data is related to text data generation. Therefore, the proposed study proposes a well-known deep learning technique for facet-based emotional mood classification using text data that can handle a large amount of content. Text data pre-processing uses stemming, segmentation, tokenization, case folding, and removal of stop words, nulls, and special characters. After data pre-processing, three word embedding approaches such as Assimilated N-gram Approach (ANA), Boosted Term Frequency Inverse Document Frequency (BT-IDF) and Enhanced Two-Way Encoder Representation from Transformers (E-BERT) are used to extract relevant features. The extracted features from the three different approaches are concatenated using the Feature Fusion Approach (FFA). The optimal features are selected using the Intensified Hunger Games Search Optimization (I-HGSO) algorithm. Finally, aspect-based sentiment analysis is performed using the Senti-BILSTM (Deep Aspect-EMO SentiNet) autoencoder based on the Hybrid Emotional Aspect Capsule autoencoder. The experiment was built on the yelp reviews dataset, IDMB movie review dataset, Amazon reviews dataset and the Twitter sentiment dataset. A statistical evaluation and comparison of the experimental results are conducted with respect to the accuracy, precision, specificity, the f1-score, recall, and sensitivity. There is a 99.26% accuracy value in the Yelp reviews dataset, a 99.46% accuracy value in the IMDB movie reviews dataset, a 99.26% accuracy value in the Amazon reviews dataset and a 99.93% accuracy value in the Twitter sentiment dataset.